Universal stereoscopic file format

ABSTRACT

Stereoscopic images may be represented in four coordinates where a first image is represented in three coordinates and a second image is represented of one coordinate. The brightness contrast is the property largely used in stereoscopic perception. The brightness and color of the first image is represented in three coordinates while the brightness of the second image is represented in the one coordinate. Color perception is dominated by the first image. A universal file format with four channels allows the stereoscopic images to be displayed as anaglyphs or as two full color images or as non-stereoscopic images. The anaglyphs may be rendered in three primary colors or four primary colors providing wide compatibility with traditional and specialized display apparatus. The universal file format facilitates methods to capture, display, convert, and communicate stereoscopic images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Provisional Applications Serial Nos. 60/928,520 filed of date May 10, 2007, 60/932,354 filed of date May 31, 2007, 60/994,001 filed of date Sep. 17, 2007, U.S. 60/997,931 filed of date Oct. 9, 2007, and 61/005,920 filed of date Dec. 10, 2007 all of which are incorporated herein in its entirety by reference thereto.

REFERENCE TO SEQUENCE LISTING, A TABLE, OR A COMPUTER PROGRAM LISTING COMPACT DISC APPENDIX

Not Applicable

BACKGROUND

Stereoscopic content may be distributed by broadcast television, stored on DVD's or other storage media, downloaded through the internet, transmitted by video phone, or distributed by other methods. In distributing stereoscopic content, it is desirable that the format of the stereoscopic content be compatible with many stereoscopic and non-stereoscopic display apparatus. It is also desirable that the amount of additional storage space or bandwidth required for stereoscopic content over non-stereoscopic content be minimal.

Stereoscopic images generally consist of two images which are related by a small change in the lateral perspective. When viewed through an enabling apparatus, stereoscopic images may provide the perception of stereoscopic depth. Stereoscopic content may be distributed as left and right image pairs or as anaglyph images. Stereoscopic content may also be distributed as a single image (stream) combined with a depth map (stream). The depth map may be used to generate a plurality of stereoscopic image pairs. Stereoscopic content may be compressed before distribution by methods similar to compressing non-stereoscopic content.

Anaglyphs are stereoscopic images wherein different sets of primary colors are used to render the first and second images of the stereo pair. Usually, the spectra of the first and second images do not overlap significantly. Then the first and second images may be viewed selectively using two complementary color viewing filters. The first viewing filter F₁ may be used to view the first image while the second viewing filter F₂ may be used to view the second image. The first filter substantially transmits the primary colors of the first image and blocks the primary colors of the second image. The second filter substantially transmits the primary colors of the second image and blocks the primary colors of the first image.

Anaglyphs are often rendered in three primary colors where the first image is rendered in two primary colors while the second image is rendered in one primary color. In red/cyan anaglyphs, the first image is rendered in green and blue primary colors while the second image is rendered in a red primary color. Other types of anaglyphs may include blue/yellow and green/magenta anaglyphs. Herein these anaglyphs are called three-color anaglyphs.

Distributing stereoscopic content as full-color left and right pairs may require substantial extra bandwidth or compression and de-compression processing that may inhibit the inclusion of stereoscope content in some of the distribution methods used for stereoscopic content. Distributing stereoscopic content in the anaglyph format using three primary color channels may require little extra bandwidth over non-stereoscopic content, however the anaglyph format is not sufficiently compatible with non-stereoscopic full-color display methods due to the lack of full-color information in the anaglyph format.

Some standard video formats such as the MPEG-C (Motion Picture Experts Group) format provide for an extra channel to accompany the video information. This extra channel may be used by methods in the prior art to provide a depth map video. The depth map video may be used to generate a plurality of stereoscopic views of the video content. These plurality of views may be used in autostereoscopic and other stereoscopic display methods to display stereoscopic images. Distributing stereoscopic content as a single image stream with a depth map stream allows stereoscopic information to be provided in a single channel and may be compatible with non-stereoscopic display methods. However a depth map generally does not provide for generating occluded data which occurs in one image of a stereoscopic pair of images, but not the other. The methods of including occluded data in addition to the depth map may reduce the compression ratio of the format and may require all stereoscopic displays to provide complex de-compression and image construction processing capabilities. Therefore, there is a need for alternate distribution methods which are highly compressible and compatible with many stereoscopic and non-stereoscopic display methods.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to the use of digital files, primary colors, color transformations, and special filters to store, distribute, display and view stereoscopic images in various formats without retinal rivalry.

The present invention concerns methods to store and distribute stereoscopic content in a universal file format that may be compatible with both stereoscopic display methods and non-stereoscopic display methods which may include full-color stereoscopic display methods and anaglyph stereoscopic display methods. The present invention also concerns methods to capture and display stereoscopic content distributed in the universal file format.

In one embodiment of the present invention, a digital stereoscopic image may be stored in four color channels in a digital file wherein the color information of the first image may be stored in three color coordinates {C₁,C₂,C₃} and the brightness Y_(B2) of the second image may be stored in a fourth coordinate {Y_(L)}. The color coordinates {C₁,C₂,C₃} may be of various color spaces wherein the brightness Y_(1B) of the first image may be determined approximately from the color information in the coordinates {C₁,C₂,C₃}. The four-channel format of the present embodiment may be summarized as {C₁,C₂,C₃,Y_(L)}. If a first image of a stereoscopic pair of images is represented by coordinates {A₁, . . . ,A_(r)}, the first image may be transformed into the coordinates {C₁,C₂,C₃} using a transformation G_(C). If a second image of a stereoscopic pair of images is represented by coordinates {B₁, . . . ,B_(S)}, the second image may be transformed into the coordinates {Y_(L)} using a transformation G_(Y).

For example, if the first image may be represented by red R₁, green G₁, and blue B₁ color coordinates of the sRGB color space and the second image may be represented by red R₂, green G₂, and blue B₂ color coordinates of the sRGB color space, the stereoscopic image may be stored in four channels as follows: {R₁,G₁,B₁,Y_(B2)} wherein the brightness of the second image may be determined approximately by the brightness function

Y _(B2)=(α₁ R ₂ ^(γ)+α₂ G ₂ ^(γ)+α₃ B ₂ ^(γ))^(1/γ)

where α₁ is about 0.2126, α₂ is about 0.7152, and α₃ is about 0.0722 and γ is about 2.2. The brightness of the first image may be determined approximately by the brightness function

Y _(B1)=(α₁ R ₁ ^(γ)+α₂ G ₁ ^(γ)+α₃ B ₁ ^(γ))^(1/γ.)

In another example, if the first image may be represented in coordinates {x₁,y₁,Y₁} of the CIE xyY color space and the second image may be represented in coordinates {x₂,y₂,Y₂} of the CIE xyY color space , then the stereoscopic image may be stored in four channels as follows: {x₁,y₁,Y₁,Y₂} where the brightness of the first image may be determined approximately by Y_(B1)=Y₁ and the brightness of the second image may be determined approximately by Y_(B2)=Y₂.

The four channel stereoscopic format of the present invention is also be applicable to stereoscopic video. One embodiment of the present invention provides a method to store a first video in three color video channels {C₁,C₂,C₃} and to store a brightness coordinate Y_(B2) of a second video in a fourth video channel {Y_(L)}. The color coordinates {C₁,C₂,C₃} may be of various color spaces. Then the stereoscopic video format of the present invention may be represented as {C₁,C₂,C₃,Y_(L)}. Herein the aspects of the present invention which are applicable to the image format {C₁,C₂,C₃,Y_(L)} are also applicable to the video format {C₁,C₂,C₃,Y_(L)} and vise versa unless otherwise stated or is obvious from the context. The stereoscopic format {C₁,C₂,C₃,Y_(L)} may be independent of the properties of a display apparatus for displaying stereoscopic or non-stereoscopic images.

Some embodiments of the present invention provide methods of displaying stereoscopic images stored in the format {C₁,C₂,C₃,Y_(L)} in various display formats such as two full color image formats, anaglyph formats and non-stereoscopic formats. A display apparatus of the present invention may comprise transformations G₁ and G₂ which transformation the color coordinates {C₁,C₂,C₃,Y_(L)} into the primary colors of the display apparatus. If a display apparatus has primary colors {P₁, . . . ,P_(m)} for displaying the first image and primary colors {Q₁, . . . ,Q_(n)} for displaying the second image, the transformation G₁ may transform or render the color and brightness contrast of the first image into the primary colors {P₁, . . . ,P_(m)} and the transformation G₂ may transform or render the color and/or brightness contrast of the second image into the primary colors {Q₁, . . . ,Q_(n)}. The transformations G₁ and G₂ may be selected in order that the observed brightness contrast of the first and second images are balanced for like subject matter and the observed color is representative of the colors of the subject of the stereoscopic image. The transformations G₁ and G₂ may compensate for the affects of the viewing apparatus (such as color viewing filters) on the observed color and brightness contrast of the first and second images.

In some embodiments of the present invention, a display apparatus providing three primary colors {P₁,P₂,Q₁} may display stereoscopic images stored in the {C₁,C₂,C₃,Y_(L)} format as conventional three-color anaglyphs. The transformation G₁ may transform the coordinates {C₁,C₂,C₃} into the primary colors {P₁,P₂}. The transformation G₂ may transform the coordinate {Y_(L)} into the primary color {Q₁}.

In some embodiments of the present invention, a display apparatus providing four primary colors {P₁,P₂,P₃,Q₁} may display stereoscopic images stored in the format {C₁,C₂,C₃,Y_(L)} as four-color anaglyphs. The transformation G₁ may transform the coordinates {C₁,C₂,C₃} into the primary colors {P₁,P₂,P₃}. The transformation G₂ may transform the coordinate {Y_(L)} into the primary color {Q₁}.

In some embodiments of the present invention, a display apparatus providing five or six primary colors {P₁,P₂,P₃,Q₁, . . . ,Q_(n)} may display stereoscopic images stored in the format {C₁,C₂,C₃,Y_(L)} as five or six-primary-color stereoscopic images. The transformation G₁ may transform the coordinates {C₁,C₂,C₃} into the primary colors {P₁,P₂,P₃}. The transformation G₂ may transform the coordinate {Y_(L)} into the primary colors {Q₁, . . . ,Q_(n)}. In the case of five or six primary colors, the format {C₁,C₂,C₃,Y_(L)} may be extended by methods of the present invention to comprise depth information and occluded image data in order to facilitate generating two color images by the display apparatus from the format {C₁,C₂,C₃,Y_(L)}.

Some embodiments of the present invention provides methods to display two full color stereoscopic image pairs from stereoscopic images stored in the format {C₁,C₂,C₃,Y_(L)} and extensions of this format. The second full color image may be obtained from the color of the first image and a fitting method which finds like subject matter in the first and second images. The fitting method may match the variation of the brightness of the first and second images with the possible help of sparse depth map data. The two colored images may be displayed in full color by conventional methods such as polarized images or autostereoscopic methods or side-by-side formats. The four-channel stereoscopic format of the present invention may be compatible with stereoscopic display methods which require two full-color images represented in coordinates {C₁,C₂,C₃} of the first image and coordinates {D₁,D₂,D₃} of the second image. The second full-color image coordinates {D₁,D₂,D₃} may be generated from the coordinates (C₁,C₂,C₃,Y_(L)) using methods of the prior art or using methods of the present invention.

The four-channel stereoscopic format of the present invention {C₁,C₂,C₃,Y_(L)} may be compatible with non-stereoscopic display methods. A non-stereoscopic display may display the first image using the coordinates {C₁,C₂,C₃} and discard the coordinates {Y_(L)}.

There are several methods of distributing or communicating stereoscopic content which are embodiments of the present invention. These embodiments may comprise the follow components: (1) methods to capture stereoscopic content; (2) methods to transform the stereoscopic content into the coordinates {C₁,C₂,C₃,Y_(L)}; (3) methods to compress the coordinates {C₁,C₂,C₃,Y_(L)}; (4) methods to store stereoscopic content; (5) methods to transmit or broadcast the content in the coordinates {C₁,C₂,C₃,Y_(L)}; (6) methods to receive the coordinates {C₁,C₂,C₃,Y_(L)}; (7) methods to de-compress the coordinates {C₁,C₂,C₃,Y_(L)}; and (8) methods to display the stereoscopic images.

DETAILED DESCRIPTION OF THE INVENTION

Additional information can be found in United States patent application serial number 2008/000835 by inventor Monte J. Ramstad which is incorporated in its entirety by reference hereto.

Additional information can be found in United States patent application serial number 2008/000841 by inventor Monte J. Ramstad which is incorporated in its entirety by reference hereto.

Additional information can be found in United States patent application serial number 2008/000855 by inventor Monte J. Ramstad which is incorporated in its entirety by reference hereto.

Additional information can be found in a co-pending United States patent application titled Display of Generalized Anaglyphs Without Retinal Rivalry by inventor Monte J. Ramstad which is incorporated in its entirety by reference hereto.

The Stereoscopic Brightness Coordinate

The present invention identifies the color property which may be substantially used by the visual system to process stereoscopic information. Herein this stereoscopic color property is called brightness contrast or brightness. The brightness may be considered a carrier of the brightness contrast in the image. When the brightness contrast is balanced in both eyes for like subject matter, retinal rivalry may be essentially absent from the stereo view and full stereoscopic depth (true-depth) may be perceived in a stereo view. Some embodiments of the present invention provide methods to store the stereoscopic information of a stereoscopic image in a single brightness coordinate Y_(B).

In stereoscopic images, retinal rivalry may be associated with unbalanced brightness contrast in the first and second images of the stereo view. For example in red/cyan anaglyphs, a red object, which may appear relatively bright through a red filter and may appear relatively dark through a cyan filter, may cause retinal rivalry in a stereo view. Usually the human visual system may not easily fuse a dark object observed by one eye with a bright object observed by the other eye. Instead, the user essentially experiences double vision, where the left and right eyes “see” independently, and stereoscopic perception may be much reduced or absent. (Stereoscopic perception requires the cooperation of both eyes.) The presence of retinal rivalry or unbalanced brightness contrast may be associated with less than full stereoscopic depth perception in a stereoscopic image.

In general, retinal rivalry may be observed in stereoscopic images where an edge in the subject matter is observed with more brightness contrast in one eye than in the other eye. This suggests that balancing the brightness contrast in all subject matter may eliminate retinal rivalry. Furthermore, stereoscopic fusion depends largely on the edges of objects being observed by both eyes. Herein an edge may be generalized to any type of brightness contrast in an image. In order for all edges to be equally detected by both eyes for all subject matter, it is apparently sufficient to observe balanced brightness contrast at all brightness levels in the image. Herein the definition of balanced brightness contrast in the first and second images is: the relative brightness contrast in the two views which minimizes retinal rivalry in the stereo view at substantially all brightness levels. By balancing the brightness contrast in all subject matter observed by the left and right eyes in a stereo view, full-depth perception may be perceived in the stereo view.

The brightness of a color is generally an integral of the brightness contrast. Therefore, the present invention provides a definition of brightness. Determining a condition of balanced brightness may require comparing the brightness of various colors. The relative brightness of different colors may depend on subjective evaluation by users. The relative brightness of different colors may vary from user to user and may vary over time for the same user. The relative brightness of different color hues may also depend on the size of a patch of a color hue, the surrounding color hues, and on the vision adaptation of the user.

In the prior art, the term brightness is generally used to describe an observed property of light rather than a physical property of light. In the present invention, brightness also describes an observed property of light. However, in the present invention, brightness is defined by the minimization of retinal rivalry in a stereo view rather than the property of a single image.

The brightness color property Y_(B) of the present invention is similar to the luminance coordinate Y in the CIE xyY color spaces however, the defining measurements are different. Some of the differences in the measurements of the brightness Y_(B) of the present invention and luminance Y of the CIE color models may include: (1) the brightness Y_(B) may be determined while viewing through colored filter glasses whereas the luminance Y is determined using unfiltered vision; (2) the number of primary colors contributing to the brightness Y_(B) and the luminance Y may be different; (3) the color patch size is not necessarily fixed when determining Y_(B) whereas a standard patch size is usually used to determine the luminance Y; (4) the brightness Y_(B) is defined by the observed retinal rivalry being minimized for all edges (or levels of contrast) whereas the luminance Y is defined by judging the “brightness” of a single image; (5) the spectra of primary colors contributing to the brightness Y_(B) and the luminance Y may be different especially when brightness Y_(B) takes into account the effects of the viewing filters on the primary colors; (6) the white point of the color space may be shifted in the measurement of the brightness Y_(B) due to the limited spectra of the viewing filters, (7) the brightness Y_(B) may include factors such as chromatic brightness and vision adaptation. Herein, chromatic brightness may include the enhanced brightness observed in highly saturated colors, in colors that include a narrow range of wavelengths of light, and in colors of various patch size.

Although the brightness Y_(B) and luminance Y properties are defined differently, in practice, they may describe similar properties of light. Therefore in some embodiments of the present invention, the brightness Y_(B) and the luminance Y may be used interchangeably. The present invention may utilize any method of calculating the brightness Y_(B) which may include methods which follow by analogy from methods of calculating the luminance Y which may be described in the prior art or developed in the future. Herein the brightness coordinate Y_(B) may also be called the luminance coordinate without loss of generality.

Four-Channel Stereoscopic File Format

One embodiment of the present invention provides methods to store digital stereoscopic image pairs in four color channels {C₁,C₂,C₃,Y_(L)} of a digital file where the color information of the first image may be stored in three color channels {C₁,C₂,C₃} and the brightness Y_(B2) of the second image may be stored in the fourth channel {Y_(L)}. If a first image has color coordinates {A₁, . . . A_(r)} and the second image has color coordinates {B₁, . . . ,B_(s)}, the first image may be transformed into the three color channels {C₁,C₂,C₃} using a transformation G_(C), and the second image may be transformed into the brightness channel {Y_(L)} using a transformation G_(Y) as follows:

$\left( {A_{1},{\ldots \mspace{11mu} A_{r}}} \right\} \overset{G_{C}}{\rightarrow}\left\{ {C_{1},C_{2},C_{3}} \right\}$ $\left\{ {B_{1},\ldots \mspace{11mu},B_{s}} \right\} \overset{G_{Y}}{\rightarrow}{\left\{ Y_{L} \right\}.}$

The first and second images represented in coordinates {A₁, . . . A_(r)} and coordinates {B₁, . . . ,B_(s)} respectively may ideally have balanced brightness for like subject matter which is the typical condition in which stereoscopic images are captured for example with two identical cameras. The transformation G_(C) may preserve the brightness contrast of the first image in the coordinates {C₁,C₂,C₃}. The transformation G_(Y) may preserve the brightness contrast of the second image in the coordinate {Y_(L)} (using any method of calculating brightness coordinate Y_(B2)). Then the coordinates {C₁,C₂,C₃} and {Y_(L)} may represent balanced brightness contrast for like subject matter in the stereoscopic image.

The transformation G_(C) which transforms the brightness contrast of the first image represented in the coordinates (A₁, . . . A_(r)} into the color coordinates {C₁,C₂,C₃} may be obtained as follows:

First, the coordinates {A₁, . . . A_(r)} may be transformed into linear brightness coordinates (A^(L) ₁, . . . A^(L) _(r)}:

A ^(L) _(j) =H _(Aj)(A _(j))

using functions H_(Aj). The functions H⁻¹ _(Aj) are the non-linearity functions encoded into the coordinates {A₁, . . . A_(r)} of the first image. Linear brightness coordinates may be defined as color coordinates whose contribution to the brightness of a color may be added together to obtain the total brightness of a color.

In some reference color spaces, the form of the functions H_(Aj) may be known. For example in the sRGB color space, the functions H_(Aj) may be approximated by:

H _(Aj)(A _(j))=A _(j) ^(Yj)

where γ_(j) may be about 2.2. Then the linear brightness Y^(L) _(A) of the first image may be obtained by adding the contribution of each coordinate to the brightness:

Y ^(L) _(A)=Σ_(j)μ_(j) A ^(L) _(j)=Σ_(j)μ_(j) H _(Aj)(A _(j))

where

1=Σ_(j)μ_(j).

The parameters μ_(j) may be determined from observations. In a reference color space, the values of μ_(j) may be known. For example in the sRGB color space, μ₁ may be about 0.2126, μ₂ may be about 0.7152, and μ₃ may be about 0.0722.

Second, a transformation W may be selected to transform the coordinates A^(L) _(j) into the linear coordinates given by C^(L) _(k)=H_(Ck)(C_(k)) as follows

C^(L) _(k)=Σ_(j)W_(kj)A^(L) _(j).

Preferably, the elements W_(kj) may be selected to represent the color (hue and saturation) of the coordinates {A₁, . . . A_(r)} in the coordinates {C₁,C₂,C₃} however, this is not required in order to display a balanced brightness stereoscopic image. The transformation W may be a linear transformation or a complex, non-linear function.

The linear brightness of the first image represented in the coordinates {C₁,C₂,C₃} may be approximated by

Y ^(L) _(C)=Σ_(k)ν_(k) C ^(L) _(k)=Σ_(k)ν_(k) H _(Ck)(C _(k))

where

1=Σ_(Z)ν_(k).

The parameters ν_(j) may be determined from observations. In a reference color space, the values of ν_(j) may be known. For example in the sRGB color space, ν₁ may be about 0.2126, ν₂ may be about 0.7152, and ν₃ may be about 0.0722.

In order to conserve the brightness in the first image, the elements W_(kj) may satisfy the condition

μ_(j)=λ₃Σ_(k)ν_(k)W_(kj)

where λ₃ is a scaling factor. These conditions may be satisfied by selecting elements W_(kj) which conserve the brightness of each initial color coordinate A_(j).

Third, the linear coordinates C^(L) _(k) may be transformed into the non-linear values

C _(k) =H ⁻¹ _(Ck)(C ^(L) _(k))

where the functions H⁻¹ _(Ck) may be determined by the color space of the coordinates {C₁,C₂,C₃}. A color space may use a gamma function to approximate H_(Ck) as follows:

C _(k)=(C ^(L) _(k))^(1/γk)

where the γ_(k) are properties of the color space.

The transformation G_(C) may be represented by the relationship

C _(k) =H ⁻¹ _(Ck)(Σ_(j) W _(kj) H _(Aj)(A _(j)))

or sometimes by the relationship

C _(k)=(Σ_(j) W _(kj) A _(j) ^(γj))^(1/γk).

where γ and γ_(k) are often near 2.2.

The transformation G_(Y) which transforms the brightness contrast of the second image represented in the coordinates (B₁, . . . B_(s)} into the color coordinates {Y_(L)} may be obtained as follows:

First, the coordinates {B₁, . . . B_(s)} may be transformed into linear brightness coordinates (B^(L) ₁, . . . B^(L) _(s)}:

B ^(L) _(j) =H _(Bj)(B _(j))

using functions H_(Bj). The functions H⁻¹ _(Bj) are the non-linearity functions encoded into the coordinates {B₁, . . . B_(s)} of the second image. Linear brightness coordinates may be defined as color coordinates whose contribution to the brightness of a color may be added together to obtain the total brightness of a color.

In some reference color spaces, the form of the functions H_(Bj) may be known. For example in the sRGB color space, the functions H_(Bj) may be approximated by:

H _(Bj)(B _(j))=B _(j) ^(γj)

where γ_(j) may be about 2.2. Then the linear brightness Y^(L) _(B) of the second image may be obtained by adding the contribution of each coordinate to the brightness:

Y ^(L) _(B)=Σ_(j)σ_(j) B ^(L) _(j)=Σ_(j)σ_(j) H _(Bj)(B _(j))

where

1=Σ_(j)σ_(j).

The parameters σ_(j) may be determined from observations. In a reference color space, the values of σ_(j) may be known. For example in the sRGB color space, σ₁ may be about 0.2126, σ₂ may be about 0.7152, and σ₃ may be about 0.0722.

Second, a transformation W may be selected to transform the coordinates B^(L) _(j) into the linear coordinates given by Y^(L) _(Y)=H_(Y)(Y_(L)) as follows

Y^(L) _(Y)=Σ_(j)σ_(j)B^(L) _(j).

In order to conserve the brightness in the second image, the linear brightness Y^(L) _(Y) of the second image represented in the coordinates {Y_(L)} may be set equal to the linear brightness Y^(L) _(B)

Y ^(L) _(Y) =Y ^(L) _(B)=Σ_(j)σ_(j) B ^(L) _(j)=Σ_(j)σ_(j) H _(Bj)(B _(j)).

Third, the linear coordinates Y^(L) _(L) may be transformed into the non-linear values

Y _(L) =H ⁻¹ _(Y)(Y ^(L) _(Y))=H ⁻¹ _(Y)(Σ_(j)σ_(j) H _(Bj)(B _(j)))

where the functions H⁻¹ _(Y) may be determined by the color space of the coordinates {Y_(L)}. A color space may use a gamma function to approximate H_(Y) as follows:

Y _(L)=(Y ^(L) _(L))^(1/γ)

where the γ is a property of the color space.

The transformation G_(Y) may be represented by the relationship

Y _(L) =H ⁻¹ _(Y)(Σ_(j)σ_(j) H _(Bj)(B _(j)))

or sometimes by the relationship

Y _(L)=(Σ_(j)σ_(j) B _(j) ^(γj))^(1/γ).

where γ and γ_(j) are often near 2.2.

In general, the stereoscopic brightness coordinates {Y_(C)} and {Y_(L)} of the present invention may depend on the properties of the display and viewing apparatus. However, for the purpose of distributing stereoscopic images, it may be advantageous for the coordinates {C₁,C₂,C₃,Y_(L)} to be independent of the display and viewing apparatus. Then a display apparatus may process the coordinates {C₁,C₂,C₃,Y_(L)} to render stereoscopic images with balanced brightness for the particular method of displaying and viewing the stereoscopic images.

One embodiment of the present invention provides methods to store stereoscopic images in coordinates {C₁,C₂,C₃} and {Y_(L)} which do not depend on the viewing filters or the display primary colors. In this embodiment, coordinates {C₁,C₂,C₃} and {Y_(L)} may be determined with respect to a standard reference color space which contains a set of reference primary colors of a standard display apparatus, and with respect to perfectly transmissive viewing filters (such as pure air).

Various reference color spaces may be used to store the first and second images with balanced brightness in the format {C₁,C₂,C₃,Y_(L)} of the present invention. For example, if the first image may be represented by red R₁, green G₁, and blue B₁ color coordinates of the sRGB color space and the second image may be represented by red R₂, green G₂, and blue B₂ color coordinates of the sRGB color space, the stereoscopic image may be stored in four channels as follows: {R₁,G₁,B₁,Y_(B2)} wherein the brightness of the second image may be determined approximately by the brightness function

Y _(B2)=(α₁ R ₂ ^(γ)+α₂ G ₂ ^(γ)+α₃ B ₂ ^(γ))^(1/γ)

where α₁ is about 0.2126, α₂ is about 0.7152, and α₃ is about 0.0722 and γ is about 2.2. The brightness of the first image may be determined approximately by the brightness function

Y _(B1)=(α₁ R ₁ ^(γ)+α₂ G ₁ ^(γ)+α₃ B ₁ ^(γ))^(1/γ).

In practice, the brightness Y_(B1) may not need to be calculated. The brightness Y_(B1) is shown here in order to support the claim that the brightness of the first and second images may be balanced Y_(B1)˜Y_(B2) for like subject matter.

In another example, if the first image may be represented in coordinates {x₁,y₁,Y₁} of the CIE xyY color space and the second image may be represented in coordinates {x₂,y₂,Y₂} of the CIE xyY color space. Then the stereoscopic image may be stored in four channels as follows: {x₁,y₁,Y₁,Y₂} where the brightness of the first image may be represented approximately by Y_(B1)=Y₁ and the brightness of the second image may be represented approximately by Y_(B2)=Y₂.

Displaying Stereoscope Images

A display apparatus receiving stereoscopic image in the coordinates {C₁,C₂,C₃,Y_(L)} may transform the coordinates into the primary colors of the display apparatus using transformations G₁ and G₂ which may take into account the properties of the primary colors of the display apparatus and viewing apparatus. For a display apparatus with primary colors {P₁, . . . ,P_(m)} for displaying the first image and primary colors {Q₁, . . . ,Q_(n)} for displaying the second image, the transformation G₁ may transform the coordinates {C₁,C₂,C₃} into the primary colors {P₁, . . . ,P_(m)}

$\left\{ {C_{1},C_{2},C_{3}} \right\} \overset{G_{1}}{\rightarrow}\left\{ {P_{1},\ldots \mspace{11mu},P_{m}} \right\}$

and the transformation G₂ may transform the coordinates {Y_(L)} into the primary colors {Q₁, . . . ,Q_(n)}

$\left\{ Y_{B\; 2} \right\} \overset{G_{2}}{\rightarrow}{\left\{ {Q_{1},\ldots \mspace{11mu},Q_{n}} \right\}.}$

The transformation G₁ may be selected so that the brightness Y_(P) observed in the primary colors {P₁, . . . ,P_(m)} may be balanced with the brightness Y_(C) represented in the coordinates {C₁,C₂,C₃}. For example, if the brightness Y_(P) may be approximated by

Y _(P)=(β₁ P ₁ ^(γ)′+β₂ P ₂ ^(γ)′+β₃ P ₃ ^(γ)′)^(1/γ)′,

and the brightness Y_(C) may be approximated by

Y _(C)(α₁ C ₁ ^(γ)+α₂ C ₃ ^(γ)+α₃ C ₃ ^(γ))^(1/γ,)

the transformation G₁ may be selected so that the brightness contrast represented in coordinates Y_(P) is balanced with the brightness contrast represented in the coordinate Y_(C), Y_(P)—Y_(C).

Similarly, the transformation G₂ may be selected so that the brightness Y_(Q) observed in the primary colors {Q₁, . . . ,Q_(n)} may be balanced with the brightness Y_(Y) represented in the coordinates {Y_(L)}. For example, if the brightness Y_(Q) may be approximated by

Y _(Q)=(δ₁ Q ₁ ^(γ)′+δ₂ Q ₂ ^(γ)′+δ₃ Q ₃ ^(γ)′)^(1/γ)′,

and the brightness Y_(Y) may be approximated by Y_(Y)=H_(Y)(Y_(L)). The transformation G₂ may be adjusted so that the brightness contrast represented in coordinates {Y_(Q)} is balanced with the brightness contrast represented in the coordinates {Y_(Y)}, Y_(Q)˜Y_(Y).

Some embodiments of the present provide methods to generate the color coordinates {D₁,D₂,D₃} of the second image from the coordinates {C₁,C₂,C₃,Y_(L)}. Then the transformation G₂ may transform the coordinates {D₁,D₂,D₃} into the primary colors {Q₁, . . . ,Q_(n)}

$\left\{ {D_{1},D_{2},D_{3}} \right\} \overset{G_{2}}{\rightarrow}{\left\{ {Q_{1},\ldots \mspace{11mu},Q_{n}} \right\}.}$

The brightness Y_(D) represented in the coordinates {D₁,D₂,D₃} may be approximated by

Y _(D)=(α₁ D ₁ ^(γ)+α₂ D ₂ ^(γ)+α₃ D ₃ ^(γ))^(1/γ).

The transformation G₂ may be selected so that the brightness contrast represented in coordinates {D₁,D₂,D₃} is balanced with the brightness contrast represented in the coordinates {Q₁, . . . ,Q_(n)}, Y_(Q)˜Y_(D).

Alternatively, the transformations G₁ and G₂ may be selected so that the displayed stereoscopic image satisfies the balanced brightness condition Y_(P)˜Y_(Q) which is a less restrictive condition on the transformations than accurately representing the brightness represented in the coordinates {C₁,C₂,C₃,Y_(L)}.

While the coordinates {Y_(L)} contain the brightness information of the second image, the coordinates {C₁,C₂,C₃} may contain the brightness information of the first image either directly such as when using CIE xyY coordinates, or indirectly such as when using sRGB coordinates. The coordinates {C₁,C₂,C₃} may also contain the chromaticity information of the first image either directly such as when using CIE xyY coordinates or indirectly such as when using sRGB coordinates.

One aspect of the present invention provides methods to identify that the stereoscopic format {C₁,C₂,C₃,Y_(L)} of the present invention is contained in the file by using metadata. The metadata may identify the properties of the color space or primary colors used to determine the values of the coordinates {C₁,C₂,C₃,Y_(L)}.

Methods to Display the Second Image in One Primary Color

One embodiment of the present invention is a method of displaying stereoscopic images comprising a display apparatus providing primary colors {P₁, . . . ,P_(m)} for displaying a first image and a providing primary color {Q₁} for displaying a second image. The first image may be represented in color coordinates {C₁,C₂,C₃}. The second image may be represented in color coordinates {Y_(L)}. The display apparatus may comprise a transformation G₁ which transforms the coordinates {C₁,C₂,C₃} into the primary colors {P₁, . . . ,P_(m)} and a transformation G₂ which transforms the coordinates {Y_(L)} into the primary color {Q₁} whereby the brightness displayed in the first and second images may be balanced for like subject matter in the first and second images. The coordinates {C₁,C₂,C₃,Y_(L)} may be coordinates of any color space from which the brightness of the first and second images may be obtained. This embodiment may provide methods to display three-color anaglyphs and four-color anaglyphs.

The transformations G₁ and G₂ may be summarized as follows:

$\left\{ {C_{1},C_{2},C_{3}} \right\} \overset{G_{1}}{\rightarrow}\left\{ {P_{1},\ldots \mspace{11mu},P_{m}} \right\}$ $\left\{ Y_{L} \right\} \overset{G_{2}}{\rightarrow}{\left\{ Q_{1} \right\}.}$

Herein, the first and second images represented in coordinates {C₁,C₂,C₃} and coordinates {Y_(L)} respectively may have balanced brightness for like subject matter. The coordinates {C₁,C₂,C₃} and coordinates {Y_(L)} may be of any color space from which the brightness may be obtained. The transformation G₁ may preserve the brightness contrast of the first image in the primary colors {P₁, . . . P_(m)}. The transformation G₂ may preserve the brightness contrast of the second image in the primary colors {Q₁}. Then the primary colors {P₁, . . . P_(m)} and {Q₁, . . . Q_(n)} may display balanced brightness contrast for like subject matter in the first and second images.

For example if the brightness of the coordinates {C₁,C₂,C₃} is represented by a function

Y _(C) =Y _(C)(C ₁ ,C ₂ ,C ₃),

and the brightness of the primary colors {P₁, . . . ,P_(m)} is represented by a function

Y _(P) =Y _(P)(P ₁ , . . . ,P _(m)),

then the transformation G₁ may be selected in order to balance the brightness contrast in the coordinates {C₁,C₂,C₃} and in the primary colors {P₁, . . . ,P_(m)}: Y_(P)˜Y_(C).

Similarly, if the brightness of the coordinates {Y_(L)} is represented by a function

Y _(Y) =Y _(Y)(Y _(L)),

and the brightness of the primary color {Q₁} is represented by a function

Y _(Q) =Y _(Q)(Q ₁),

then the transformation G₂ may be selected in order to balance the brightness contrast in the coordinates {Y_(L)} and in the primary color {Q₁}: Y_(Q)˜Y_(Y).

Since stereoscopic images stored in the format {C₁,C₂,C₃,Y_(L)} may be created to satisfy the condition Y_(C)˜Y_(Y), the additional conditions of Y_(Q)˜Y_(Y) and Y_(P)˜Y_(C) may provide for balance brightness in the display primary colors, Y_(Q)˜Y_(P). Thus, stereoscopic images distributed in the coordinates {C₁,C₂,C₃,Y_(L)} may be displayed with balanced brightness in the first and second images. In general, the relative brightness of the primary colors {P₁, . . . ,P_(m)} and {Q₁} may be adjusted in order to compensate for the transmission spectra of specific viewing filters.

Herein the values of the primary colors {P₁, . . . ,P_(m)} and {Q₁} may be considered the color coordinate values of a display apparatus. The display apparatus may transform the coordinate values into the physical primary colors {P₁, . . . ,P_(m)} and {Q₁}. Herein the notation does not always distinguish between the color coordinates of the display apparatus and the physical primary colors of a display apparatus although the distinction will be clear from the context to one skilled in the art.

The transformation G₁ which transforms the brightness contrast of the first image represented in the coordinates {C₁,C₂,C₃} into the values of the primary colors {P₁, . . . ,P_(m)} may be obtained as follows:

First, the coordinates {C₁,C₂,C₃} may be transformed into linear brightness coordinates {C^(L) ₁,C^(L) ₂,C^(L) ₃}:

C ^(L) _(j) =H _(Cj)(C _(j))

using functions H_(Cj). The functions H⁻¹ _(Cj) are the non-linearity functions encoded into the coordinates {C₁,C₂,C₃} of the first image. Linear brightness coordinates may be defined as color coordinates whose contribution to the brightness of a color may be added together to obtain the total brightness of a color.

In some reference color spaces, the form of the functions H_(Cj) may be known. For example in the sRGB color space, the functions H_(Cj) may be approximated by:

H _(Cj)(C _(j))=C _(j) ^(γj)

where γ_(j) may be about 2.2. Then the linear brightness Y^(L) _(C) of the first image may be obtained by adding the contribution of each coordinate to the brightness:

Y ^(L) _(C)=Σ_(j)α_(j) C ^(L) _(j)=Σ_(j)α_(j) H _(Cj)(C _(j))

where

1=Σ_(j)α_(j).

The parameters α_(j) may be determined from observations. In a reference color space, the values of α_(j) may be known. For example in the sRGB color space, α₁ may be about 0.2126, α₂ may be about 0.7152, and α₃ may be about 0.0722.

Second, a transformation M may be selected to transform the coordinates C^(L) _(j) into the linear primary color values P^(L) _(k)=H_(P)(P_(k)) of the display apparatus as follows

P ^(L) _(k)=Σ_(j) M _(kj) C ^(L) _(j)=Σ_(j) M _(kj) H _(Cj)(C _(j)).

Preferably, the elements M_(kj) may be selected to represent the chromaticity of the coordinates {C₁,C₂,C₃} in the primary colors {P₁, . . . ,P_(m)} however, this is not required in order to display a balanced brightness stereoscopic image. In general, the transformation M may be a linear transformation or a complex, non-linear function.

The linear brightness of the first image displayed in the primary colors {P₁, . . . ,P_(m)} may be approximated by

Y ^(L) _(P)=Σ_(k)β_(k) P _(L) ^(k)=Σ_(k)β_(k) H _(P)(P _(k))

where

1=Σ_(k)β_(k).

The parameters β_(k) may be determined from experimental observations of the relative brightness of the primary colors of the display apparatus viewed through a first viewing filter F₁.

In order to conserve the brightness in the first image, the elements M_(kj) may satisfy the condition

α_(j)=λ₁Σ_(k)β_(k) M _(kj)

where λ₁ is a scaling factor. These conditions may be satisfied by selecting elements M_(kj) which conserve the brightness of each initial color coordinate C_(j).

Third, the linear primary colors P^(L) _(k) may be transformed into the non-linear values used by the display apparatus

P _(k) =H ⁻¹ _(Pk)(P ^(L) _(k))

where the functions H⁻¹ _(Pk) may be determined for the primary colors of a display apparatus. A display apparatus may use a gamma function to approximate H_(Pk) as follows:

P _(k)=(P ^(L) _(k))^(1/γk)

where the γ_(k) are properties of the display apparatus.

The transformation G₁ may be represented by the relationship

P _(k) =H ⁻¹ _(Pk)(Σ_(j) M _(kj) H _(Cj)(C _(j)))

or sometimes by the relationship

P _(k)=(Σ_(j) M _(kj) C _(j) ^(γj))^(1/γk).

where γ_(j) and γ_(k) are often near 2.2.

Similarly, the transformation G₂ of the second image into the primary color {Q₁} which conserves the brightness contrast represented in the coordinates {Y_(L)} may be obtained as follows:

First, the coordinates {Y_(L)} may be transformed into linear brightness coordinates {Y^(L) _(L)}:

Y ^(L) _(L) =H ^(Y)(Y _(L))

using function H_(Y). The function H⁻¹ _(Y) is the non-linearity function encoded into the coordinates {Y_(L)} of the second image.

In some reference color spaces, the form of the function H_(Y) may be known. For example in the sRGB color space, the functions H_(Y) may be approximated by:

H _(Y)(Y _(L))=Y _(L) ^(γY)

where γ_(Y) may be about 2.2. Then the linear brightness Y^(L) _(Y) of the second image may be represented in the coordinates {Y_(L)} may be given by:

Y^(L) _(Y)=Y_(L) ^(γY).

Second, a transformation may be selected to transform the coordinates Y^(L) _(L) into the linear primary color value Q^(L) ₁=H_(Qk)(Q_(k)) of the display apparatus as follows

Q ^(L) ₁ =αY ^(L) _(L) =αH _(Y)(Y _(L))

where α is a constant.

Third, the linear primary colors Q^(L) ₁ may be transformed into the non-linear values used by the display apparatus

Q ₁ =H ⁻¹ _(Q1)(Q ^(L) ₁)

where the functions H⁻¹ _(Q1) may be determined for the primary colors of a display apparatus. A display apparatus may use a gamma function to approximate H_(Q1) as follows:

Q ₁=(Q ^(L) ₁)^(1/γQ)

where the γ₁ is a property of the display apparatus.

The transformation G₂ may be represented by the relationship

Q _(k)=α_(Q) H ⁻¹ _(Q1)(H _(Y)(Y _(L)))

or sometimes by the relationship

Q₁=α_(Q)Y_(L) ^(Y)

where α_(Q) may be a selectable parameter, γ=γ_(Y)/γ_(Q), and γ_(Y) and γ_(Q) are often near 2.2.

The filters F₁ and F₂ may be colored filters or polarized filters. In some display methods, viewing filters may not be needed and the F₁ and F₂ filters may be replaced by the viewing apparatus of the display method. If the spectra of the primary colors {P₁, . . . ,P_(m)} do not substantially overlap the spectra of the primary color {Q₁}, the viewing filters may be color filters where the first viewing filter F₁ substantially transmits the primary colors {P₁, . . . ,P_(m)} and blocks the primary colors {Q₁, . . . ,Q_(n)}; and the second viewing filter F₂ substantially transmits the primary colors {Q₁, . . . ,Q_(n)} and blocks the primary colors {P₁, . . . ,P_(m)}.

The two transformations G₁ and G₂ may be used to transform the initial stereoscopic coordinates {C₁,C₂,C₃,Y_(L)} containing any distribution of color into displayed anaglyph images with balanced brightness contrast. These methods apply to anaglyphs displayed using any number of primary colors. Examples are described below for two and three primary colors {P₁, . . . ,P_(m)} and one primary color {Q₁} corresponding to three-color and four-color anaglyphs.

Display of Four-Color Anaglyphs

This first example demonstrates a method to display stereoscopic images distributed in the coordinates {C₁,C₂,C₃,Y_(L)} as a four-color anaglyph with balanced brightness contrast.

As one example of the transformation G₁, a first image represented by red, green, and blue color coordinates {R,G,B}={C₁,C₂,C₃} may be transformed into red, green, and blue display primary colors {P₁,P₂,P₃}. For this example, the coordinates {R,G,B} may be coordinates of the sRGB color space. For a particular display apparatus, the linear brightness of the primary colors may be approximated by

Y ^(L) _(P)=0.30P ^(L) ₁+0.60P ^(L) ₂+0.10P ^(L) ₃.

The linear brightness of the first image in the coordinates {R,G,B} of the sRGB color space may be approximated by the luminance function

Y ^(L) _(C)=0.21R ^(L) ₁+0.72G ^(L) ₂+0.07B ^(L) ₃.

The conditions on the transformation M_(kj) may be

0.21=0.30M _(1R)+0.60M _(2R)+0.10M _(3R),

0.72=0.30M _(1G)+0.60M _(2G)+0.10M _(3G), and

0.07=0.30M _(1B)+0.60M _(2B)+0.10M _(3B).

In a simple case, the off diagonal elements of M_(kj) may be set to zero. Then

0.21=0.30M_(1R),

0.72=0.60M_(2G), and

0.07=0.10M_(3B).

It follows that M_(1R)=0.7, M_(2G)=1.2, and M_(2B)=0.7. Then

P^(L) ₁=0.7 R^(L),

P^(L) ₂=1.2 G^(L), and

P^(L) ₃=0.7 B^(L).

Since M_(2G)=1.2 is greater than 1.0, the values of P^(L) ₂ may exceed the capabilities of the display device. Therefore, the elements M_(kj) may be divided by 1.2 to obtain

P^(L) ₁=0.58 R^(L),

P^(L) ₂=1.0 G^(L), and

P^(L) ₃=0.58 B^(L).

The renormalization of the elements of M_(kj) is consistent with balancing the brightness contrast of the displayed images. The primary values Q₁ may also be reduced by a factor of 1.2, however the scale of Q₁ relative to the P_(j) may be selectable.

The complete transformation G₁ may be written as:

P ₁=(0.58 R ^(2.2))^(1/γ),

P ₂ =G ^(2.2/γ), and

P ₃=(0.58 B ^(2.2))^(1/γ)

where γ may often be chosen to be identical for each primary color P_(j). The value of γ may depend on the properties of the primary colors of the display apparatus. If γ=2.2, then

P₁=0.78R,

P₂=G, and

P₃=0.78B.

In general, a transformation G₁ may be selected to also represent the chromaticity of the first image in the primary colors {P₁,P₂,P₃} to the chromaticity of the first image in the coordinates {R,G,B}. In this case, the off-diagonal elements of M_(kj) may not necessarily be zero and the transformation may be more complex.

For a particular display apparatus, the transformation G₂ may transform the coordinates {Y_(L)} into the primary color {Q₁} as follows:

Q₁=α_(Q)Y_(L)

where α_(Q) may be chosen to select the relative brightness of the second image; and where γ=2.2 in the range of brightness values selected for the primary color {Q₁}.

Display of Three-Color Anaglyphs

A second example demonstrates a method to display stereoscopic images distributed in the coordinates {C₁,C₂,C₃,Y_(L)} as red/cyan anaglyphs with balanced brightness contrast.

As one example of the transformation GI, a first image represented by red, green, and blue color coordinates {R,G,B}={C₁,C₂,C₃} may be transformed into green, and blue display primary colors {P₁,P₂}. For this example, the coordinates {R,G,B} may be coordinates of the sRGB color space. For a particular display apparatus, the linear brightness of the primary colors may be approximated by

Y ^(L) _(P)=0.80P ^(L) ₁+0.20P ^(L) ₂.

The linear brightness of the first image in the coordinates {R,G,B} of the sRGB color space may be approximated by the luminance function

Y ^(L) _(C)=0.21R ^(L) ₁+0.72G ^(L) ₂+0.07B ^(L) ₃.

The balanced brightness conditions on the transformation M_(kj) may be

0.21=0.80M _(2R)+0.20M _(3R),

0.72=0.80M _(2G)+0.20M _(3G), and

0.07=0.80M _(2B)+0.20M _(3B).

If the primary color P₁ is green and the primary color P₂ is blue, a simple case may map the green coordinate G 100% into the green primary color P₁, the blue coordinate B 100% into the blue primary color P₂, and the red coordinate R 50% into each of the green and blue primary colors P₁ and P₂ . In other words, M_(3G)=M_(2B)=0 and M_(2R)=M_(3R). Then

0.21=0.80M _(2R)+0.20M _(3R),

0.72=0.80M_(2G), and

0.07=0.20M_(3B).

It follows that M_(2R)=M_(3R)=0.21, M_(2G)=0.9, and M_(2B)=0.35. Then

P^(L) ₁=0.9 G ^(L)+0.21 R ^(L) and

P ^(L) ₂=0.35 B ^(L)+0.21 R ^(L).

Since M_(2G)+M_(2R)=1.11, is greater than 1.0, the values of P^(L) ₁ may exceed the capabilities of the display apparatus. Therefore, the M_(kj) may be divided by 1.11 to obtain

P ^(L) ₁=0.81 G ^(L)+0.19 R ^(L) and

P _(L) ₂=0.32 B ^(L)+0.19 R _(L).

The renormalization of the elements M_(kj) is consistent with balancing the brightness contrast of the displayed images. The primary values Q₁ may also be reduced by a factor of 1.11, however the scale of Q₁ relative to the Pj may be selectable.

The complete transformation G₁ may be written as:

P ₁=(0.81 G ^(2.2)+0.19 R ^(2.2) )^(1/γ,)

P ₂=(0.32 B ^(2.2)+0.19 R ^(2.2))^(1/γ)

where in this case γ may be 2.2.

In general, a transformation G₁ may be selected to also represent the chromaticity of the first image in the primary colors {P₁,P₂}. For example, red hue may be selectively displayed as yellow hues, gray hue, or purple hues. In other words, the M_(2R) and M_(3R) and M_(kj) in general may vary throughout an image as long as they satisfy the balanced brightness constraints at each point. This allows for complex projections of the red coordinate R into the primary colors P₁ and P₂. These complex projections are within the scope of the present invention.

For a particular display apparatus, the transformation G₂ may transform the coordinates {Y_(L)} into the primary color {Q₁} as follows:

Q₁=α_(Q)Y_(L)

where α_(Q) may be chosen to select the relative brightness of the second image; and where γ=2.2 in the range of brightness values selected for the primary color {Q₁}.

This second example demonstrates a method to display stereoscopic images distributed in the {C₁,C₂,C₃,Y_(L)} format as red/cyan anaglyphs with balanced brightness contrast. Similarly stereoscopic images distributed in the {C₁,C₂,C₃,Y_(L)} format may be displayed as blue/yellow and green/magenta anaglyphs with balanced brightness by methods of the present invention.

Display of Color Stereoscopic Image Pairs

One embodiment of the present invention provides methods to transform the coordinates {C₁,C₂,C₃,Y_(L)} into two color images. The coordinates {C₁,C₂,C₃} may be transformed using a transformation G₁ into a first set of primary colors {P₁, . . . ,P_(m)} of a display apparatus. The coordinates {C₁,C₂,C₃,Y_(L)} may be transformed using a transformation G₂ into a second set of primary colors {Q₁, . . . ,Q_(m)} of a display apparatus. Transformations G₁ and G₂ may be summarized as follows:

$\left\{ {C_{1},C_{2},C_{3}} \right\} \overset{G_{1}}{\rightarrow}\left\{ {P_{1},\ldots \mspace{11mu},P_{m}} \right\}$ $\left\{ {C_{1},C_{2},C_{3}} \right\} \overset{G_{2}}{\rightarrow}{\left\{ {Q_{1},\ldots \mspace{11mu},Q_{n}} \right\}.}$

The transformation G₂ may transform the brightness coordinate Y_(Y)=Y_(L) into each primary color Q_(j). Or a transformation G_(D) may transform the coordinates {C₁,C₂,C₃}{Y_(L)} into color coordinates {D₁,D₂,D₃} of the second image which may be of the same color space as the coordinates {C₁,C₂,C₃} of the first image. Then the transformation G₂ may transform the coordinates {D₁,D₂,D₃} into the primary colors {Q₁, . . . ,Q_(n)}

${\left\{ {C_{1},C_{2},C_{3}} \right\} \left\{ Y_{L} \right\}}\overset{G_{D}}{\rightarrow}{\left\{ {D_{1},D_{2},D_{3}} \right\} \overset{G_{2}}{\rightarrow}{\left\{ {Q_{1},\ldots \mspace{11mu},Q_{n}} \right\}.}}$

The transformation G₂ which transforms the brightness Y_(D) of the second image represented in the coordinates {D₁,D₂,D₃} into the values of the primary colors {Q₁, . . . ,Q_(n)} may be obtained as follows:

First, the coordinates {D₁,D₂,D₃} may be transformed into linear brightness coordinates {D^(L) ₁,D^(L) ₂,D^(L) ₃}:

D ^(L) _(j) =H _(Dj)(D _(j))

using functions H_(Dj). The functions H⁻¹ _(Dj) are the non-linearity functions encoded into the coordinates {D₁,D₂,D₃} of the second image. Linear brightness coordinates may be defined as color coordinates whose contribution to the brightness of a color may be added together to obtain the total brightness of a color.

In some reference color spaces, the form of the functions H_(Dj) may be known. For example in the sRGB color space, the functions H_(Dj) may be approximated by:

H _(Dj)(D _(j))=D ^(γj)

where the γ_(j) may be about 2.2. Then the linear brightness Y^(L) _(D) of the second image represented in coordinates {D₁,D₂,D₃} may be obtained by adding the contribution of each coordinate to the brightness:

Y ^(L) _(D)=Σ_(j)δ_(j) D ^(L) _(j)=Σ_(j)δ_(j) H _(Dj)(D _(j))

where

1=Σ_(j)δ_(j).

The parameters δ_(j) may be determined from observations. In a reference color space, the values of δ_(j) may be known. For example in the sRGB color space, δ₁ may be about 0.2126, δ₂ may be about 0.7152, and δ₃ may be about 0.0722.

Second, a transformation N may be selected to transform the coordinates D^(L) _(j) into the linear primary color values Q^(L) _(k) given by Q^(L) _(k)=H^(Qk)(Q_(k)) as follows

Q ^(L) _(k)=Σ_(j) N _(kj) D ^(L) _(j)=Σ_(j) N _(kj) H _(Qk)(Q _(k)).

Preferably, the elements N_(kj) may be selected to represent the chromaticity of the coordinates {D₁,D₂,D₃} in the primary colors {Q₁, . . . ,Q_(n)} however, this is not required in order to display a balanced brightness stereoscopic image. In general, the transformation N may be a linear transformation or a complex, non-linear function.

The linear brightness Y^(L) _(Q) of the first image displayed in the primary colors {Q₁, . . . ,Q_(n)} may be approximated by

Y ^(L) _(Q)=Σ_(k)ε_(k) Q _(L) ^(k)=Σ_(k)ε_(k) H _(Qk)(Q _(k))

where

1=Σ_(k)ε_(k).

The parameters ε_(k) may be determined from experimental observations of the relative brightness of the primary colors of the display apparatus viewed through a second viewing filter F₂.

In order to conserve the brightness in the first image, the elements N_(kj) may satisfy the condition

δ_(j)=λ₂Σ_(k)ε_(k)N_(kj)

where λ₂ is a scaling factor. These conditions may be satisfied by selecting elements N_(kj) which conserve the brightness of each initial color coordinate D_(j).

Third, the linear primary colors Q^(L) _(k) may be transformed into the non-linear values used by the display apparatus

Q _(k) =H ⁻¹ _(Qk)(Q ^(L) _(k))

where the functions H⁻¹ _(Qk) may be determined for the primary colors of a display apparatus. A display apparatus may use a gamma function to approximate H_(Qk) as follows:

Q _(k)=(Q ^(L) _(k))^(1/γk)

where the γ_(k) are properties of the display apparatus.

The transformation G₂ may be represented by the relationship

Q _(k) =H ⁻¹ _(Qk)(Σ_(j) N _(kj) H _(Dj)(D _(j)))

or sometimes by the relationship

Q _(k)=(Σ_(j) N _(kj) D _(j) ^(γj))^(1/γk).

where γ_(j) and γ_(k) are often near 2.2.

The first and second sets of primary colors may have different polarization states, different color spectra, or may be displayed on different regions of surfaces, or may be displayed time sequentially, or may be displayed autostereoscopically. In general, the primary color coordinates {Q₁, . . . ,Q_(n)} of the second image may be generated from both the values of coordinates {Y_(L)} of the second image and the values of coordinates {C₁,C₂,C₃} of the first image using either methods of the prior art or methods described herein. Some methods may use additional coordinates such as a depth map or chromaticity values of the second image which may be included in the file format of the present invention.

One embodiment of the present invention provide a method of generating a color second image from the coordinates {C₁,C₂,C₃}{Y_(L)} by combining the brightness of the second image with the chromaticity of the first image. The coordinates {C₁,C₂,C₃} may be transformed using transformation G_(D) into coordinates {x₁,y₁,Y₁} of the CIE xyY color space. Then the second color image may be constructed from the coordinates {x₁,y₁,Y_(L)} using transformation G_(D). Then the coordinates {x₁,y₁,Y_(L)} may be transformed into the primary colors {Q₁, . . . ,Q_(n)} using transformation G₂. In this case, the hues of the second generated image may show parallax artifacts due to the parallax difference in the hues {x₁,y₁} and brightness {Y_(L)} in the generated second image. In general, a depth map may be used to reduce the parallax artifacts in the generated second image.

Use of a Depth Map

Some stereoscopic display apparatus require colored first and second images. These display apparatus may display two full color images from stereoscopic images in the format {C₁,C₂,C₃,Y_(L)} by using methods of the present invention to generate the color of the second image from the color of the first image. A depth map may be used to facilitate generating the color of the second image from the color of the first image. In addition to generating the color of the second image, a depth map may be used to generate a plurality of stereoscopic views for use in auto-stereoscopic displays or in order to select or change the range of stereoscopic parallax in a displayed stereoscopic view.

A depth map {Z₂} may be used to calculate or obtain the parallax between the first and second images. Herein, a depth map may have either the depth values of the subject or the parallax values of the subject since the depth and parallax may be related by simple geometry. The color of a point in the second image may be determined from the color of the points in the first image which are offset from the point in the second image by an amount of parallax using a transformation G_(D).

It is often convenient for the depth map to be in registry with the second image (to be of the second image). Then the parallax value used for a point in the second image may be obtained from the corresponding point in the depth map. The depth map coordinates and image coordinates may be written as {C₁,C₂,C₃,Y₂,Z₂}.

A depth map may be generated from the coordinates {C₁,C₂,C₃,Y_(L)} by using methods of the present invention or by using methods of the prior art. A depth map may be generated from the luminance or brightness coordinates of the first and second images, and from additional depth information which may be included in the file using a transformation G_(M). First, it may be convenient to transform the coordinates {C₁,C₂,C₃} of the first image into brightness and chromaticity coordinates

{C₁,C₂,C₃,Y_(L)}→{x₁,y₁,Y₁,Y₂}.

Then the transformation G_(M) may be used to generate the depth map coordinates

$\left\{ {x_{1},y_{1},Y_{1},Y_{2}} \right\} \overset{G_{M}}{\rightarrow}{\left\{ {x_{1},y_{1},Y_{1},Y_{2},Z_{2}} \right\}.}$

Then the chromaticity of the second image may be obtained from the chromaticity of the first image using transformation G_(D)

$\left\{ {x_{1},y_{1},Y_{1},Y_{2},Z_{2}} \right\} \overset{G_{D}}{\rightarrow}\left. \left\{ {x_{1},y_{1},Y_{1},x_{2},{y_{2}Y_{2}},} \right\}\rightarrow\left\{ {C_{1},C_{2},C_{3},D_{1},D_{2},D_{3}} \right\} \right.$

where the coordinates {x₁,y₁,Y₁}{x₂,y₂,Y₂,} may be transformed into coordinates {C₁,C₂,C₃} and {D₁,D₂,D₃} of the original color space.

In the transformation G_(M), the variation of the luminance of the second image may be fit to the variation of the luminance of the first image using variable parallax values and a fitting method. A fitting algorithm may be used to find the parallax values which best match the variation of the first and second images over certain regions of the first and second images. The fitting method may be performed at more than one length scale in order to determine the reliability of the fit over a certain region. Generally, a fitting method may be used to generate a depth map over the parts of the subject in the second image which are also in the first image. Regions of the second image that are occluded in the first image may not generate meaningful depth map points.

A fitting method may generate errors in the depth map at certain points. Generally, the more complex the fitting method the fewer the errors in the depth map. However, a complex fitting method may be computationally demanding which may cause an undesirable delay the display of the images. The errors in a generated depth map may be resolved or filled in using depth coordinates {p} which may be included in the file format of the present invention {C₁,C₂,C₃,Y_(L)}{p}. In particular, points {q} that generate errors in a simple fitting method may be resolved (over ridden) with depth data {p} which may be included in the file for those points. Both the depth values {p} and the point locations {q} may be included in the file format {C₁,C₂,C₃,Y_(L)}{p,q} of the present invention. A fitting method may continue generating the depth map using the depth map data as a new starting point. In this way, by the inclusion of a small amount of depth or parallax information in the file format of the present invention, a complete and accurate depth map may be generated using a simple or fast fitting method in transformation G_(M)

${\left\{ {x_{1},y_{1},Y_{1},Y_{2}} \right\} \left\{ {p,q} \right\}}\overset{G_{M}}{\rightarrow}\left\{ {x_{1},y_{1},Y_{1},Y_{2},Z_{2}} \right\}$

In transformation G_(M), it may not be necessary for the depth map {Z₂} or {p,q} to contain accurate depth information. Instead, the depth map need only be usable to obtain the correct color of the second image. In this case, an error in the depth map may be defined as a point where the generated color of the second image may be inaccurate. The coordinates {p,q} may be included in the format to resolve these errors. Additional errors in the depth map may be defined for convenience in order to reduce the time required to generate the depth map {Z₂}. For example, very large values of parallax or large changes in parallax obtained by a fitting algorithm may be defined as errors in the depth map. Then by excluding large changes in parallax in a fitting algorithm may allow the depth map {Z₂} to be generated more simply or more quickly.

In general, a depth map may be included in the {C₁,C₂,C₃,Y_(L)} file format of the present invention. The depth map may be of low resolution or may be sparsely filled. The depth map may contain data where a particular method for generating a depth map from the first and second images may be known to generate errors or may be inefficiently generated. The sparse set of data points may be sufficient to provide clues to a simple fitting method in order to eliminate errors in the generated depth map. The sparse depth map may comprise points at sharp changes in depth, at the borders of occluded data, and at other locations. These sparse depth map points may be compressed by methods of the prior art. The depth map data points may be stored in units of parallax between the left and right images.

For example, a simple fitting method may start with pairs of point locations q₁ and q₂ and parallax values p₁ and p₂ which may be provided in the file in two channels {q,p}. The method for obtaining the parallax values pi for the points q_(i) between q₁ and q₂ may be linear interpolation as follows:

p _(i) =p ₁(q ₂ −q _(i))/(q ₂ −q ₁)+p ₂(q _(i) −q ₁)/(q ₂ −q ₁)

where the interpolation may be used for q_(i) between q₁ and q₂. In another example, a curved parallax between points q₁ and q₂ may be obtained as follows:

p _(i)=σ_(i) +p ₁(q ₂ −q _(i))/(q ₂ −q ₁)+p ₂(q _(i) −q ₁)/(q ₂ −q ₁)

where Y′₁ is the difference from a linear interpolation. The values of σ_(i) may be determined for each point from Y ₂(q _(i))=Y′ ₁(q _(i) +p _(i)) where Y′₁ is the scaled luminance of the first image and Y₂ is the luminance of the second image and where the interpolation is valid for q_(i) between q₁ and q₂. Here either Y₁ or Y₂ may be scaled to match the scale of the other by using the luminance values at the depth map data points as follows:

Y′ ₁ =Y ₂(q ₁ +p ₁)+(Y ₁(q ₁)−Q ₁(q ₁))(Y ₂(q ₂ +p ₂)−Y ₂(q ₁ +p ₁))/(Y ₁(q ₂)−Y ₁(q ₁))

Since the luminance values are defined on discrete points, an interpolation between points of the Y₁ values may be helpful as follows:

Y ₂(q _(i))=p″ _(i) Y′ ₁(q _(i) +p′ _(i)+1)+(1−p″ _(i))Y′ ₁(q _(i) +p′ _(i)),

p _(i) =p′ _(i) +p″ _(i)

where p′_(i) is the integer part of p_(i) and p″_(i) is the fractional part of p_(i). The task of a fitting method to obtain a depth map may be to search for values of p_(i) or p′_(i) and p″_(i) for which Y₂=Y′₁.

This fitting method may generate errors if performed over large distances or over sharp changes in depth or over occluded regions. However, these errors may be anticipated when the file is constructed so that the depth data pairs {q₁,p₁,q₂,p₂} may be sufficient to avoid errors. Sufficient depth data pairs may be included to cover all regions of the second image which are not occluded in the first image. If the density of depth data pairs required to avoid errors is above a certain level in a region of the second image, the region of the second image may be considered to be occluded in the first image. In the occluded regions, the depth data points may be omitted from the file. In this way, the density of depth information in the file may be controlled. If a depth map is provided on a regular grid of points in the image, the coordinates {q} may be omitted.

For the regions of the second image which are not occluded in the first image, the color data of the second image may be determined with using transformation G_(F) by the following method:

X ₂(q _(i))=p″ _(i) X ₁(q _(i) +p′ _(i)+1)+(1−p″ _(i))X ₁(q _(i) +p′ _(i))

where X is a color property provided for the first image in the format {C₁,C₂,C₃,Y_(L)}.

The coordinates {C₁,C₂,C₃,Y_(L)} do not contain the color information for portions of the second image occluded in the first image. Furthermore the use of a depth map may not be sufficient to generate the color of the second image in regions that are occluded in the first image. However in the transformation G_(F), the missing color of the second image may be filled in by extrapolation or partial extrapolation. In partial extrapolation, the missing color may be filled in by extrapolation where the color is gradually reduced to grayscale as the distance into the occluded area increases.

On the other hand, the color of the second image in the occluded regions may be included in separate channels. These extra channels may be lightly populated with color data since only a small fraction of the second image will usually be occluded in the first image. Furthermore the chromaticity information for the occluded regions may be provided at a resolution smaller than the resolution of the coordinates {Y_(L)}. The extra channels in the occluded regions may contain the chromaticity coordinates {x₂,y₂} of the second image. Then the format of the present invention may be written as {C₁,C₂,C₃,Y_(L)}{x₂,y₂}{p,q}. The transformation G_(D) may use the color coordinates {x₂,y₂} for points in the second image which are not obtained accurately or efficiently using other methods. In general, it is expected that the chromaticity data provided for the occluded regions may be compressed to a higher degree than the combination of the luminance and the chromaticity of the occluded regions.

Compressing the Brightness Channels

Providing a sparse depth map in the four-channel format of the present invention has another advantage. The depth map may be used by a compression method to compress the Y₁ and Y_(L) channels of the first and second images respectively. On one embodiment of the present invention, a difference coordinate may be generated

dY=Y ₂ −Y ₁.

Then the coordinate dY may be compressed. In another embodiment of the present invention, a depth map may be used to make the difference coordinate generally smaller as follows:

dY′=Y ₂(q _(i))−Y ₁(q _(i) +p _(i))

where p_(i) may be provided by the depth map at low resolution or at sparsely populated points. Usually, coordinates {dY′} may be compressed to a higher degree than coordinates {dY}.

In another embodiment of the present invention, stereoscopic images may be represented using coordinates {C₁,C₂,C₃,dY} where the coordinates {dY} provide information from which coordinates {Y_(L)} may be obtained using a transformation G_(H). The coordinates {C₁,C₂,C₃,dY} may be compressed in a file. Herein, some aspects of the present invention which are applicable to the coordinates {C₁,C₂,C₃,Y_(L)} are also applicable to the coordinates {C₁,C₂,C₃,dY}. The transformation G_(H) may be written as Y₂=Y₁+dY.

In another embodiment of the present invention, stereoscopic images may be represented using coordinates {C₁,C₂,C₃,dY′}{p,q} where the coordinates {dY′} and {p,q} provide information from which coordinates {Y_(L)} may be obtained using a transformation G_(H). The coordinates {C₁,C₂,C₃,dY′}{p,q} may be compressed in a file. Herein, some aspects of the present invention which are applicable to the coordinates {C₁,C₂,C₃,Y_(L)} are also applicable to the coordinates {C₁,C₂,C₃,dY′}{p,q}. The transformation G_(H) may be written as Y₂(q_(i))=Y₁(q_(i)+p_(i))+dY′ where p_(i) may be a value of p obtained from the coordinates {p,q}.

In summary, the four-channel stereoscopic format of the present invention {x₁,y₁,Y₁,Y_(L)} may comprise two additional channels {x₂,y₂} which contain the chromaticity of the second image in the regions occluded in the first image, and extra channels {q,p} which contain sparsely populated depth data pairs or an extra channel {p} which contains a low resolution depth map. Then the complete format may be represented as

{C₁,C₂,C₃,Y_(L)}{x₂,y₂}{q,p},

or

{C₁,C₂,C₃,Y_(L)}{x₂,y₂}{p}.

The color coordinates may be representable as xyY coordinates. Then the formats of the present invention may represented as

{x₁,y₁,Y₁,Y_(L)}{x₂,y₂}{q,p},

or

{x₁,y₁,Y₁,Y_(L)}{x₂,y₂}{p}.

The coordinates {x₂,y₂}{p} or {x₂,y₂}{q,p} may be used to generate the full-color second image. Other methods of the prior art for generating the full-color second image may be included as a component of some embodiments of the present invention. The coordinates {Y_(L)} may be represented as difference coordinates {dY} or {dY′}. Then the formats of the present invention may represented as for example

{x₁,y₁,Y₁,dY′}{x₂,y₂}{q,p},

or

{x₁,y₁,Y₁,dY′}{x₂,y₂}{p}.

Herein the aspects of the present invention which apply to the formats containing coordinates {C₁,C₂,C₃,Y_(L)} also apply to formats containing coordinates {C₁,C₂,C₃,dY} or {C₁,C₂,C₃,dY′} as will be obvious to one skilled in the art.

Some standard file formats such as JPEG, PNG, and MPEG use compression methods to reduce the size of the digital files containing images or video. One aspect of the present invention uses these or other compression methods to reduce the bandwidth or file size of the stereoscopic formats of the present invention. A stereoscopic image contained in a compressed file format of the present invention may require little additional bandwidth or storage space compared with non-stereoscopic content.

Stereoscopic Communication

There are several methods of distributing or communicating stereoscopic content which are embodiments of the present invention. These embodiments may comprise the follow components: (1) methods to capture stereoscopic content; (2) methods to transform the stereoscopic content into the coordinates {C₁,C₂,C₃,Y_(L)}; (3) methods to compress the coordinates {C₁,C₂,C₃,Y_(L)}; (4) methods to store stereoscopic content; (5) methods to transmit or broadcast the content in the coordinates {C₁,C₂,C₃,Y_(L)}; (6) methods to receive the coordinates {C₁,C₂,C₃,Y_(L)}; (7) methods to de-compress the coordinates {C₁,C₂,C₃,Y_(L)}; and (8) methods to display the stereoscopic images.

Stereoscopic images may be captured by a stereoscopic camera and stored or transmitted in the stereoscopic format {C₁,C₂,C₃,Y_(L)} of the present invention.

Another embodiment of the present invention provides a method to capture stereoscopic content into the format {C₁,C₂,C₃,Y_(L)}. This embodiment may comprise a stereoscopic camera similar in design to any stereoscopic camera of the prior art. For example the camera apparatus may comprise a first objective lens optically cooperative with a first image sensor and a second objective lens optically cooperative with a second image sensor. The first and second lenses may be positioned apart by the distance d of the stereo base. A stereo base is the distance between the first and second objective lenses in a stereoscopic camera. The optical axes of the first and second lenses may be generally directed toward the common subject. The axes may be generally parallel or the axes may intersect at a small angle at a distance from the lenses in a direction toward the common subject. The angle between the axes may be adjustable. The distance between the first and second lenses may be adjustable. The first and second sensor images may be periodically sampled to capture a first and second image of stereoscopic pairs of images. The first sensor may capture a full color first image in for example three primary color coordinates {A₁,A₂,A₃}₂ . The second sensor may capture a full color second image in for example three primary color coordinates {B₁,B₂,B₃} or the second sensor may capture a grayscale second image in for example one primary coordinate {B₁} which represents the brightness contrast in the second image. The camera apparatus may comprise a method to transform the full-color first images into the coordinates {C₁,C₂,C₃} of a reference color space using a transformation G_(C). The camera apparatus may comprise a method to transform the second images into brightness coordinates {Y_(L)} using a transformation G_(Y). The transformations G_(C) and G_(Y) may be selected to preserve the brightness contrast of the first and second images in the coordinates {C₁,C₂,C₃,Y_(L)}. The camera apparatus may comprise a DVD, hard disk, memory card or other storage device for storing the captured images. The camera apparatus may comprise a communication port to transfer the stereoscopic images to a display apparatus or to an external storage device. The camera apparatus may comprise a display apparatus for displaying the captured stereoscopic images.

A stereoscopic player of the present invention may be used to play stereoscopic content stored in the stereoscopic format {C₁,C₂,C₃,Y_(L)} of the present invention.

Another embodiment of the present invention provides a method to display stereoscopic content comprising: a display apparatus; a DVD, hard drive or other storage apparatus for storing a stereoscopic content; and a player which may read the stereoscopic content from the storage device. The stereoscopic content may be stored in the format {C₁,C₂,C₃,Y_(L)} of the present invention. If the stereoscopic images are compressed, a decompression function implemented in software or hardware may be used to decompress the stereoscopic images. The stereoscopic content in the format {C₁,C₂,C₃,Y_(L)} may be displayed using the primary colors {P₁, . . . P_(m),Q₁, . . . ,Q_(n)} of the display apparatus where the spectra of the primary colors {Q₁, . . . ,Q_(n)} may substantially not overlap the spectra of the primary colors {P₁, . . . ,P_(m)}. The coordinates {C₁,C₂,C₃} of the stereoscopic images may be transformed into the primary colors {P₁, . . . ,P_(m)} using a transformation G₁ while the coordinates {Y_(L)} may be transformed into the primary colors {Q₁, . . . ,Q_(n)} using a transformation G₂. The transformations G₁ and G₂ may be selectable so that the brightness contrast of the first and second images may be balanced when viewed through viewing filters of the present invention used for viewing anaglyph images.

Stereoscopic content received by a stereoscopic converter apparatus in the format {C₁,C₂,C₃,Y_(L)} may be converted to the primary color coordinates {T₁, . . . ,T_(t)} of a display apparatus for either non-stereoscopic display or stereoscopic display. The stereoscopic converter apparatus may communicate with a display device through a communication port.

Another embodiment of the present invention provides methods to receive stereoscopic content in the format {C₁,C₂,C₃,Y_(L)} and transform the coordinates into primary color coordinates {T₁, . . . ,T_(t)} of a display apparatus and transmit the color coordinates to a display apparatus. The present embodiment may comprise a receiver for receiving the stereoscopic content in the format {C₁,C₂,C₃,Y_(L)}, transformations G₁ and G₂ for transforming the first and second images into primary color coordinates {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} for stereoscopic display where the set of primary colors {T₁, . . . T_(t)} may be equal to the union of the set of primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)}. The coordinates {C₁,C₂,C₃} may be transformed into the primary colors {P₁, . . . ,P_(m)} using the transformation G₁ while the coordinates {Y_(L)} may be transformed into the primary colors {Q₁, . . . ,Q_(n)} using transformation G₂. The transformations G₁ and G₂ may be selectable so that the brightness contrast of the first and second images may be balanced when viewed through viewing filters F₁ and F₂ of the present invention used for viewing anaglyph images. The present embodiment may further comprise a transformation G₅ for transforming the first image into the primary color coordinates {T₁, . . . ,T_(t)} for non-stereoscopic display. The present embodiment may further comprise a transmitter that may transmit the color coordinates {T₁, . . . ,T_(t)} or {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} to a display apparatus. The present embodiment may further comprise a communication port for transmitting the color coordinates {T₁, . . . ,T_(t)} or {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} to a display apparatus. The methods of transmission of the transmitter may comprise electronic or electromagnetic signal transmission or other signal transmission method. The receiver may receive stereoscopic content by electronic or electromagnetic signal or other transmission method.

Stereoscopic communication between two users each with a stereoscopic communication device of the present invention may be performed using the stereoscopic format {C₁,C₂,C₃,Y_(L)} of the present invention.

Another embodiment of the present invention provides methods for two-way communication of stereoscopic images comprising: a first communication apparatus which may be used by a first user; a stereoscopic camera apparatus capable of capturing stereoscopic content in the format {C₁,C₂,C₃,Y_(L)}; a method to compress images in the format {C₁,C₂,C₃,Y_(L)} into a standard compressed format such an mpeg or jpeg format; a method to transmit the compressed stereoscopic video or images to a second communication device which may be used by a second user at a distance location. The methods of transmission may comprise internet transmission (wireless or wire connection), cell phone transmission, any TV broadcast transmission or satellite transmission, phone line, or TV cable transmission or any other method used for remote communication. The first communication apparatus may also comprise a method to receive stereoscopic images in the format {C₁,C₂,C₃,Y_(L)} from a remote source. The images received from a remote source may be decompressed by the communication apparatus if necessary. The received video or still images may be displayed by the display apparatus using primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)}. The stereoscopic content {C₁,C₂,C₃,Y_(L)} may be displayed using the primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} of a display apparatus where the spectra of the primary color {Q₁, . . . ,Q_(n)} may substantially not overlap the spectra of the primary colors {P₁, . . . ,P_(m)}. The coordinates {C₁,C₂,C₃} may be transformed into the primary colors {P₁, . . . ,P_(m)} while the coordinates {Y_(L)} may be transformed into the primary colors {Q₁, . . . ,Q_(n)}. The transformations G₁ and G₂ may be selectable so that the brightness contrast of the first and second images may be balanced when viewed through viewing filters of the present invention used for viewing anaglyph images. The display apparatus may be attachable to the communication device through a communication port.

As noted above, the present invention is applicable to digital files, primary colors, color transformations and special filters and is believed to be particularly useful for storing, distributing, displaying and viewing stereoscopic images in various formats without retinal rivalry. The present invention should not be considered limited to the particular examples described above, but rather should be understood to cover all aspects of the invention as fairly set out in the attached claims. Various modifications, equivalent processes, as well as numerous structures to which the present invention may be applicable will be readily apparent to those of skill in the art to which the present invention is directed upon review of the present specification. The claims are intended to cover such modifications and devices. 

1. A method of representing stereoscopic images comprising: a first image representable as coordinates {A₁, . . . ,A_(r)}; a second image representable as coordinates {B₁, . . . ,B_(s); transforming coordinates {A₁, . . . ,A_(r)} into color coordinates {C₁,C₂,C₃} using transformation G_(C); transforming coordinates {B₁, . . . ,B_(s)} into coordinates {Y_(L)} using transformation G_(Y); brightness Y_(C) of said first image is represented in coordinates {C₁,C₂,C₃}; brightness Y_(Y) of said second image is represented in coordinates {Y_(L)}; and wherein the brightness contrast of said first and second images represented in coordinates {C₁,C₂,C₃,Y_(L)} are balanced for like subject matter in said first and second images.
 2. The method of claim 1 wherein linear brightness Y^(L) _(A) represented in coordinates {A₁, . . . ,A_(r)} is given by Y ^(L) _(A)=Σ_(j)μ_(j) H _(Aj)(A _(j)) where μ_(j) are constants; linear brightness Y^(L) _(C) represented in coordinates {C₁,C₂,C₃} is given by Y ^(L) _(C)=Σ_(k)ν_(k) H _(Ck)(C _(k)) where ν_(k) are constants; G_(C) is represented by relationship C_(k)=H⁻¹ _(Ck)(Σ_(j)W_(kj)H_(Aj)(A_(j))); and W_(kj) satisfies condition μ_(j)=Σ_(k)ν_(k)W_(kj) where λ₃ is a constant.
 3. The method of claim 2 wherein H_(Aj)(A_(j))=A_(j) ^(γj) and H_(Ck)(C_(k))=C_(k) ^(γk) where γ_(j) and γ_(k) are constants whereby G_(C) is represented by the relationship C _(k)=(Σ_(j) W _(kj) A _(j) ^(γj))^(1/γk).
 4. The method of claim 1 wherein linear brightness Y^(L) _(B) represented in coordinates {B₁, . . . ,B_(s)} is given by Y ^(L) _(B)=Σ_(j)σ_(j) H _(Bj)(B _(j)) where σ_(j) are constants; linear brightness Y^(L) _(Y) represented in coordinates {Y_(L)} is given by Y ^(L) _(Y) =H _(Y)(Y _(L)); and G_(Y) is represented by relationship Y_(L)=H⁻¹ _(Y)(Σ_(j)σ_(j)H_(Bj)(B_(j))).
 5. The method of claim 4 wherein H_(Bj)(B_(j))=B_(j) ^(γj) and H_(Y)(Y_(L))=Y_(L) ^(Y) where γ_(j) and γ are constants whereby G_(Y) is represented by the relationship Y _(L)=(Σ_(j)σ_(j) B _(j) ^(γj))^(1/γ).
 6. The method of claim 1 wherein coordinates {C₁,C₂} are chromaticity coordinates; and coordinates {C₃} are brightness coordinates.
 7. The method of claim 1 wherein coordinates {C₁,C₂,C₃} are primary color coordinates.
 8. A method of displaying stereoscopic images comprising: a first image represented by coordinates {C₁,C₂,C₃}; a second image represented by coordinates {Y_(L)}; primary colors {P₁, . . . ,P_(m)} for displaying said first image; primary colors {Q₁} for displaying said second image; transforming coordinates {C₁,C₂,C₃} into primary colors {P₁, . . . ,P_(m)} using transformation G₁; transforming coordinates {Y_(L)} into primary color {Q₁} using transformation G₂; brightness Y_(P) of said first image represented in primary colors {P₁, . . . ,P_(m); brightness Y_(Q) of said second image represented in primary color {Q₁}; and wherein the brightness contrast of said first and second images represented in primary colors {P₁, . . . ,P_(m),Q₁} are balanced for like subject matter in said first and second images.
 9. The method of claim 8 wherein: linear brightness Y^(L) _(C) represented in coordinates {C₁,C₂,C₃} is given by Y ^(L) _(C)=Σ_(j)α_(j) H _(Cj)(C _(j)) where α_(j) are constants; linear brightness Y^(L) _(P) represented in primary colors {P₁, . . . ,P_(m)} is given by Y ^(L) _(P)=Σ_(k)β_(k) H _(Pk)(P _(k)) where ⊖_(k) are constants; G₁ is represented by relationship P_(k)=H⁻¹ _(Pk)(Σ_(j)M_(kj)H_(Cj)(C_(j))); and M_(kj) satisfies condition α_(j)=λ₁Σ_(k)β_(k)M_(kj) where λ₁ is a constant.
 10. The method of claim 9 wherein H_(Cj)(C_(j))=C_(j) ^(γj) and H_(Pk)(P_(k))=P_(k) ^(γk) where γ_(j) and γ_(k) are constants whereby G1 is represented by relationship P _(k)=(Σ_(j) M _(kj) C _(j) ^(γj))^(1/γk).
 11. The method of claim 8 wherein linear brightness Y^(L) _(Y) represented in coordinates {Y_(L)} is given by Y _(L) ^(Y) =H _(Y)(Y _(L)); linear brightness Y^(L) _(Q1) represented in primary color {Q₁} is given by Q ^(L) ₁ =H _(Q1)(Q ₁); and G₂ is represented by relationship Q_(k)=α_(Q)H⁻¹ _(Q1)(H_(Y)(Y_(L))) where α_(Q) is a constant.
 12. The method of claim 11 wherein H_(Y)(Y_(L))=Y_(L) ^(γY) and H_(Q1)(Q₁)=Q₁ ^(γQ) where γ_(Y) and γ_(Q) are constants whereby G₂ is represented by the relationship Q₁=α_(Q)Y_(L) ^(γ) where γ is a constant.
 13. The method of claim 8 further comprising: a first filter F₁ substantially transmitting primary colors {P₁, . . . ,P_(m)} and blocking primary color {Q₁}; and a second filter F₂ substantially transmitting primary colors {Q₁} and blocking primary colors {P₁, . . . ,P_(m)}.
 14. The method of claim 8 wherein the set of spectra of primary colors {P₁,P₂;P₃} are not substantially overlapping the spectra of primary colors {Q₁}.
 15. A method of displaying stereoscopic images comprising: a first image represented by coordinates {C₁,C₂,C₃}; a second image represented by coordinates {Y_(L)}; primary colors {P₁, . . . ,P_(m)} for displaying said first image; primary colors {Q₁, . . . ,Q_(n)} for displaying said second image; transforming coordinates {C₁,C₂,C₃} into primary colors {P₁, . . . ,P_(m)} using transformation G₁; transforming coordinates {Y_(L)} into primary colors {Q₁, . . . ,Q_(n)} using transformation G₂; brightness Y_(P) of said first image represented in primary colors {P₁, . . . ,P_(m); brightness Y_(Q) of said second image represented in primary colors {Q₁, . . . ,Q_(n)}; and wherein the brightness contrast of said first and second images represented in primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} are balanced for like subject matter in said first and second images.
 16. The method of claim 15 wherein: linear brightness Y^(L) _(C) represented in coordinates {C₁,C₂,C₃} is given by Y ^(L) _(C)=Σ_(j)α_(j) H _(Cj)(C _(j)) where α_(j) are constants; linear brightness Y^(L) _(P) represented in primary colors {P₁, . . . ,P_(m)} is given by Y ^(L) _(P)=Σ_(k)β_(k) H _(Pk)(P _(k)) where β_(k) are constants; G₁ is represented by relationship P_(k)=H⁻¹P_(k)(Σ_(j)M_(kj)H_(Cj)(C_(j))); and M_(kj) satisfies condition α_(j)=λ₁Σ_(k)β_(k)M_(kj) where λ₁ is a constant.
 17. The method of claim 16 wherein H_(Cj)(C_(j))=C_(j) ^(γj) and H_(Pk)(P_(k))=P_(k) ^(γk) where γ_(j) and γ_(k) are constants whereby P _(k)=(Σ_(j) M _(kj) C _(j) ^(γj))^(1/γk).
 18. The method of claim 15 further comprising: transforming coordinates {C₁,C₂,C₃,Y_(L)} into coordinates {x₁,y₁,Y_(L)} using transformation G_(D) where x₁ and y₁ are chromaticity coordinates of said first image, and Y_(L) represents the brightness of said second image; transforming coordinates {x₁,y₁,Y_(L)} into primary colors {Q₁, . . . ,Q_(n)} using transformation G₂; and wherein the brightness contrast of said second image is displayed in primary colors {Q₁, . . . ,Q_(n)} and chromaticity coordinates {x₁,y₁} are represented in primary colors {Q₁, . . . ,Q_(n)}.
 19. The method of claim 15 further comprising sparse coordinates {q} or sparse coordinates {p,q} where coordinates {p} are depth or parallax coordinates associated with said second image and coordinates {q} are location coordinates associated with said second image.
 20. The method of claim 19 further comprising: transforming coordinates {C₁,C₂,C₃,Y_(L)}{p,q} into coordinates {x₁,y₁,Y₁,Y₂} using transformation G_(D) where x₁ and y₁ are chromaticity coordinates of said first image, Y₁ approximates brightness Y_(C) of said first image represented in coordinates {C₁,C₂,C₃}, and Y₂ approximates brightness Y_(Y) of said second image represented in coordinates {Y_(L)}; transforming coordinates {x₁,y₁,Y₁,Y₂}{d₁,d₂} into coordinates {x₂,y₂,Y₂} using transformation G_(D) where x₂ and y₂ are chromaticity coordinates of said second image; transforming coordinates {x₂,y₂,Y_(B2)} into primary colors {Q₁, . . . ,Q_(n)} using transformation G₂; and where the brightness contrast of said second image Y_(Y) is displayed in primary colors {Q₁, . . . ,Q_(n)} and chromaticity coordinates {x₂,y₂} are represented in primary colors {Q₁, . . . ,Q_(n)}.
 21. The method of claim 19 further comprising transforming coordinates {C₁, C₂,C₃,Y_(L)}{p,q} into coordinates {C₁,C₂,C₃,Y_(L),D₂} using transformation G_(M) where coordinates {D₂} are depth or parallax values for each coordinate {Y_(L)} of said second image which is not occluded in said first image.
 22. The method of claim 15 further comprising chromaticity coordinates {x₂,y₂} for a set of points in said second image.
 23. The method of claim 15 further comprising: transforming coordinates {C₁,C₂,C₃,Y_(L)} into coordinates {C₁,C₂,C₃,D₁,D₂,D₃} using transformation G_(D) where coordinates {D₁,D₂,D₃} represent said second image in the color space of coordinates {C₁,C₂,C₃}; linear brightness Y^(L) _(D) represented in coordinates {D₁,D₂,D₃} is given by Y ^(L) _(D)=Σ_(j)δ_(j) H _(Dj)(D _(j)) where δ_(j) are constants; linear brightness Y^(L) _(Q) represented in primary colors {Q₁, . . . ,Q_(n)} is given by Y ^(L) _(Q)=Σ_(k)ε_(k) H _(Qk)(Q _(k)) where ε_(k) are constants; G₁ is represented by relationship Q_(k)=H⁻¹Q_(k)(Σ_(j)N_(kj)H_(Dj)(D_(j))); and N_(kj) satisfies condition δ_(j)=λ₂Σ_(k)ε_(k)N_(kj) where λ₂ is a constant.
 24. The method of claim 23 wherein H_(Dj)(D_(j))=D_(j) ^(γj) and H_(Qk)(Q_(k))=Q_(k) ^(γk) where γ_(j) and γ_(k) are constants whereby G₁ is represented by relationship Q _(k)=(Σ_(j) N _(kj) D ^(γ) _(j))^(1/γk).
 25. The method of claim 15 further comprising: a first filter F₁ substantially transmitting primary colors {P₁, . . . ,P_(m)} and blocking primary color {Q₁, . . . Q_(n)}; and a second filter F₂ substantially transmitting primary colors {Q₁, . . . ,Q_(n)} and blocking primary colors {P₁, . . . ,P_(m)}.
 26. A method of representing stereoscopic images comprising: a first image representable as coordinates {A₁, . . . ,A_(r)}; a second image representable as coordinates {B₁, . . . ,B_(s)}; transforming coordinates {A₁, . . . ,A_(r)} into color coordinates {C₁,C₂,C₃} using transformation G_(C); transforming coordinates {B₁, . . . ,B_(s)} into coordinates {dY′} using transformation G_(Y); brightness coordinates {Y₁} of said first image represented in coordinates {C₁,C₂,C₃}; coordinates {dY′} representing the difference in brightness coordinates {Y₂} of said second image represented in coordinates {C₁,C₂,C₃} at points {q} and brightness coordinates {Y′₁} of said first image at points {q+p} offset from the points {q} in said second image by parallax coordinates {p}, dY′=Y₂−Y′₁; and wherein the brightness contrast of said first and second images represented in coordinates {C₁,C₂,C₃,dY′} are balanced for like subject matter in said first and second images.
 27. The method of claim 26 further comprising transforming coordinates {C₁,C₂,C₃,dY′} into coordinates {C₁,C₂,C₃,Y₂} using transformation G_(H).
 28. The method of claim 26 further comprising sparse coordinates {q} or sparse coordinates {p,q} where coordinates {p} are depth or parallax coordinates associated with said second image and coordinates {q} are location coordinates associated with said second image.
 29. The method of claim 26 wherein coordinates {p} are equal to zero or a constant.
 30. A method of capturing stereoscopic images comprising: a first objective lens; a second objective lens, said first lens spaced apart from said second lens by distance d; at least one image sensor for capturing images; capturing a first image into coordinates {C₁,C₂,C₃}; capturing a second image into coordinates {Y_(L)}; brightness Y_(C) of said first image represented in coordinates {C₁,C₂,C₃}; brightness Y_(Y) of said second image represented in coordinates {Y_(L)}; and wherein the brightness contrast of said first and second images represented in coordinates {C₁,C₂,C₃,Y_(L)} are balanced for like subject matter in said first and second images.
 31. The method of claim 30 further comprising storing coordinates {C₁,C₂,C₃,Y_(L)} in a storage apparatus.
 32. The method of claim 30 further comprising: a display apparatus providing primary colors {P₁,P₂,P₃,Q₁}; transforming coordinates {C₁,C₂,C₃} into primary colors {P₁, . . . ,P_(m)} using transformation G₁; transforming coordinates {Y_(L)} into primary colors {Q₁} using transformation G₂; and wherein the brightness contrast of said first and second images displayed in primary colors {P₁, . . . ,P_(m),Q₁} are balanced for like subject matter in said first and second images.
 33. The method of claim 30 further comprising: a communication port; and transferring coordinates {C₁,C₂,C₃,Y_(L)} to the display apparatus through said communication port.
 34. A method of storing stereoscopic images comprising: a storage apparatus for storing stereoscopic images; a first image represented in coordinates {C₁,C₂,C₃}; a second image represented in coordinates {Y_(L)}; brightness Y_(C) of said first image represented in coordinates {C₁,C₂,C₃}; brightness Y_(Y) of said first image represented in coordinates {Y_(L)}; and wherein the brightness contrast of said first and second images represented in coordinates {C₁,C₂,C₃,Y_(L)} are balanced for like subject matter in said first and second images.
 35. The method of claim 34 further comprising: a display apparatus providing primary colors {P₁,P₂;P₃} and primary colors {Q₁}; transforming coordinates {C₁,C₂,C₃} into primary colors {P₁,P₂;P₃}; and transforming coordinates {Y_(L),C₂,C₃} into primary color {Q₁}.
 36. The method of claim 35 wherein the set of spectra of primary colors {P₁,P₂;P₃} are not substantially overlapping the set of spectra of primary colors {Q₁}.
 37. The method of claim 35 further comprising: a first filter F₁ substantially transmitting primary colors {P₁, . . . ,P_(m)} and blocking primary color {Q₁, . . . ,Q_(n)}; and a second filter F₂ substantially transmitting primary colors {Q₁, . . . ,Q_(n)} and blocking primary colors {P₁, . . . ,P_(n)}.
 38. A method of converting stereoscopic images comprising: a receiver apparatus; receiving said stereoscopic images in format {C₁,C₂,C₃,Y_(L)} using said receiver apparatus; transforming coordinates {C₁,C₂,C₃} into primary color coordinates {P₁, . . . ,P_(m)} using transformation G₁; and transforming coordinates {C₁,C₂,C₃,Y_(L)} into primary color coordinates {Q₁, . . . ,Q_(n)} using transformation G₂.
 39. The method of claim 38 further comprising: a transmitter apparatus; and transmitting color coordinates {P₁, . . . ,P_(n),Q₁, . . . ,Q_(n) using said transmitter apparatus.
 40. The method of claim 38 further comprising: a display apparatus providing primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} and displaying color coordinates {P₁, . . . ,P_(n),Q₁, . . . ,Q_(n)} using said display apparatus.
 41. The method of claim 38 wherein: brightness Y_(P) of said first image is represented in the coordinates {P₁, . . . ,P_(m)}; brightness Y_(Q) of said second image is represented in the coordinates {Q₁, . . . ,Q_(n)}; and wherein the brightness contrast of said first and second images represented in coordinates {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} are balanced for like subject matter in said first and second images.
 42. The method of claim 38 wherein n=1.
 43. The method of claim 42 wherein m=2.
 44. The method of claim 38 wherein n=0 whereby stereoscopic images are converted to non-stereoscopic images represented in coordinates {P₁, . . . ,P_(m)};
 45. The method of claim 38 further comprising: a transmitter apparatus; transforming coordinates {C₁,C₂,C₃} into primary color coordinates {T₁, . . . ,T_(t)} using transformation G₅; and transmitting color coordinates {T₁, . . . ,T_(t)} using said transmitter apparatus.
 46. The method of claim 45 wherein t=m+n.
 47. A method of two-way communication of stereoscopic images comprising: a stereoscopic camera apparatus; capturing stereoscopic image pairs into coordinates {C₁,C₂,C₃,Y_(L)}; said first images of stereoscopic image pairs captured in coordinates {C₁,C₂,C₃}; the brightness Y_(C) of said first images represented in coordinates {C₁,C₂,C₃}; said second images of stereoscopic image pairs captured in coordinates {Y_(L)}; brightness Y_(Y) of said second images represented in coordinates {Y_(L)}; transmitting the coordinates {C₁,C₂,C₃,Y_(L)} of the captured image pairs using a transmitter apparatus; receiving stereoscopic image pairs in coordinates {C₁,C₂,C₃,Y_(L)} using a receiver apparatus; a display apparatus providing primary colors {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)}; transforming coordinates {C₁,C₂,C₃} of the received image pairs into primary color coordinates {P₁, . . . ,P_(m)} using transformation G₁; and transforming coordinates {C₁,C₂,C₃,Y_(L)} of the received images into primary color coordinates {Q₁, . . . ,Q₁} using transformation G₂.
 48. The method of claim 47 wherein n=1.
 49. The method of claim 48 wherein n=2.
 50. The method of claim 47 wherein: brightness Y_(P) of the first image is represented in coordinates {P₁, . . . ,P_(m)}; brightness Y_(Q) of the second image is represented in coordinates {Q₁, . . . ,Q_(n)}; and wherein the brightness contrast of said first and second images represented in coordinates {P₁, . . . ,P_(m),Q₁, . . . ,Q_(n)} are balanced for like subject matter in said first and second images.
 51. A method of transmitting stereoscopic images comprising: a transmitter apparatus; first images of stereoscopic image pairs represented in coordinates {C₁,C₂,C₃}; the brightness Y_(C) of said first images represented in coordinates {C₁,C₂,C₃}; second images of stereoscopic image pairs represented in coordinates {Y_(L)}; the brightness Y_(Y) of said second images represented in coordinates {Y_(L)}; and transmitting coordinates {C₁,C₂,C₃,Y_(L)} of said image pairs using the transmitter apparatus. 